Skip to content

fix: validate CI env and isolate check types#253

Merged
zeitlinger merged 28 commits intomainfrom
codex/ci-env-required
Apr 30, 2026
Merged

fix: validate CI env and isolate check types#253
zeitlinger merged 28 commits intomainfrom
codex/ci-env-required

Conversation

@zeitlinger
Copy link
Copy Markdown
Member

@zeitlinger zeitlinger commented Apr 28, 2026 •
edited
Loading

Summary

Validates GitHub-related environment at runtime for checks that need it, and moves check-specific setup/runtime behavior behind registry-owned check-type metadata.

Runtime behavior:

  • CI lychee runs fail when GITHUB_TOKEN is missing, including --full mode.
  • CI renovate-deps runs fail when both GITHUB_COM_TOKEN and GITHUB_TOKEN are missing.
  • Local lychee and renovate-deps runs warn when token env is missing, instead of failing.
  • GitHub Actions PR runs in changed-file mode fail when lychee PR link-remap metadata is missing.
  • --full and non-CI runs do not require lychee PR link-remap metadata.
  • renovate-deps forwards GITHUB_TOKEN to Renovate as GITHUB_COM_TOKEN when needed.

Registry/init refactor:

  • Checks can attach a CheckTypeDef for shared type-level behavior across multiple registry entries, such as biome and biome-format.
  • Native checks now use the NativeCheck / PreparedNativeCheck path and CheckKind::Native, replacing the older special-check model.
  • flint init invokes selected check-type init hooks once per check type.
  • Current init-hook owners: renovate-deps, rumdl, ryl, taplo, cargo-fmt, biome, and biome-format.
  • Check-specific setup logic is owned by src/linters modules; the central init flow only builds context and invokes selected hooks.
  • Registry metadata also owns adaptive relevance, flint linters status overrides, baseline triggers, non-verbose failure output normalization, missing toolchain component hints, and generated workflow setup needs.
  • New flint.toml gets [checks.renovate-deps] when renovate-deps is selected; existing config is patched only to preserve migrated v1 exclude_managers values.
  • Docs now refer to in-process checks as native checks instead of special checks.

Also extracts shared CI/token env helpers and updates check-specific docs.

Validation

  • cargo fmt --check
  • cargo check
  • cargo test runner::
  • cargo test init::
  • cargo test flint_setup
  • cargo test lychee
  • cargo test renovate_deps
  • cargo test linters::env
  • cargo test generate_
  • cargo test readme_linter_table_in_sync
  • cargo test names_prefer_binary_or_native_command
  • cargo test linters_table_matches_fixture_without_fake_bins
  • cargo test linter_status_reports_no_binary_and_not_configured
  • cargo test adaptive_checks_declare_relevance_hooks
  • cargo test editorconfig_checker_json_is_optional_not_generated_baseline
  • FLINT_CASES=general/init-rust cargo test cases
  • FLINT_CASES=general/init-idempotent cargo test cases
  • FLINT_CASES=general/fast-only-explicit-override cargo test cases
  • FLINT_CASES=lychee/local-missing-token-warns cargo test cases
  • FLINT_CASES=renovate-deps/ci-missing-token cargo test cases
  • FLINT_CASES=renovate-deps/local-missing-token-warns cargo test cases
  • FLINT_CASES=renovate-deps/fast-only-relevant cargo test cases
  • FLINT_CASES=renovate-deps/fast-only-irrelevant cargo test cases
  • git diff --check
  • pre-push hook: mise run lint:fix (cargo run -q --bin flint -- run --fix)

Note: full cargo test is not runnable in this local environment because registry::tests::all_registry_binaries_found requires all registered linter binaries on PATH. Full E2E cases also need real external tools such as biome, ruff, renovate, shfmt, ryl, and rumdl.

@zeitlinger
fix: validate CI env for GitHub link remaps
dc2d8fa 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger zeitlinger force-pushed the codex/ci-env-required branch from bb0d8fa to dc2d8fa Compare April 29, 2026 05:28
@zeitlinger
docs: clarify GitHub token scope
1cd214b 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
docs: clarify lychee GitHub token scope
61205f3 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
docs: keep lychee docs specific
8d01922 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
fix: validate renovate GitHub token env
69a2849 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
fix: patch renovate config during init
d753269 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: move renovate init config helper
abc56e4 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
fix: limit renovate init config patching
88472ca 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: run selected linter init hooks
12cf57b 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger zeitlinger changed the title fix: fail CI link checks when required env is missing fix: validate GitHub env for CI checks Apr 29, 2026
@zeitlinger
refactor: move linter-specific hooks to registry
934f7c2 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
style: apply clippy suggestion
8389717 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
style: format adaptive hook check
b8c5127 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
test: expect renovate token warning in explicit fast run
127e515 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger zeitlinger changed the title fix: validate GitHub env for CI checks fix: validate CI env and isolate linter hooks Apr 29, 2026
zeitlinger added 14 commits April 29, 2026 07:55
@zeitlinger
refactor: move biome init hook to linter module
7d1535d 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: move renovate init logic to linter module
deb0fe8 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: move remaining init hooks to linter modules
9a4de14 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
fix: run shared init hooks once
c3848d4 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: keep init hook identity with linters
809db77 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: model linter-owned hooks
c4ab468 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: keep special binaries with linters
a347334 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: extract special linter metadata
4b6d286 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: execute special linters via trait
8233840 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: drop unused special kinds
b46eb63 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: move special config display to linter
bf8c0fc 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: replace special kind with setup flag
86a498e 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: use explicit fix builder
b14a0d3 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
refactor: rename linter model to check type
fa51fdf 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger zeitlinger changed the title fix: validate CI env and isolate linter hooks fix: validate CI env and isolate check types Apr 29, 2026
@zeitlinger zeitlinger marked this pull request as ready for review April 29, 2026 11:19
@zeitlinger zeitlinger requested a review from a team as a code owner April 29, 2026 11:19
Copilot AI review requested due to automatic review settings April 29, 2026 11:19
Copilot AI reviewed Apr 29, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors Flint’s check registry to support “native” (in-process) checks with shared check-type metadata, and adds runtime CI environment validation for GitHub-dependent checks (notably lychee and renovate-deps).

Changes:

  • Introduces CheckType/NativeCheck infrastructure to move type-level behavior (init hooks, adaptive relevance, status overrides, baseline triggers, output normalization, workflow setup hints) into registry-owned metadata.
  • Adds CI-vs-local token/PR-metadata validation and clearer warnings/errors for lychee and renovate-deps, plus forwarding GITHUB_TOKEN → GITHUB_COM_TOKEN for Renovate when needed.
  • Updates init flow, docs, and E2E cases to align with the new native-check model and CI env requirements.

Reviewed changes

Copilot reviewed 63 out of 63 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/e2e.rs Clears additional GitHub-related env vars to make E2E runs deterministic.
tests/cases/renovate-deps/up-to-date/test.toml Sets GITHUB_TOKEN env for renovate-deps case.
tests/cases/renovate-deps/up-to-date-renovaterc-json/test.toml Sets GITHUB_TOKEN env for renovate-deps case.
tests/cases/renovate-deps/out-of-date/test.toml Ensures renovate-deps case has GITHUB_TOKEN env and updates expected output formatting.
tests/cases/renovate-deps/local-missing-token-warns/test.toml Adds new case verifying local missing-token behavior warns but still runs into snapshot error.
tests/cases/renovate-deps/local-missing-token-warns/files/renovate.json5 Adds minimal renovate config fixture for local-missing-token case.
tests/cases/renovate-deps/local-missing-token-warns/files/mise.toml Adds renovate tool fixture for local-missing-token case.
tests/cases/renovate-deps/fix-update/test.toml Sets GITHUB_TOKEN env for renovate-deps fix-update case.
tests/cases/renovate-deps/fix-create/test.toml Sets GITHUB_TOKEN env for renovate-deps fix-create case.
tests/cases/renovate-deps/fast-only-relevant/test.toml Sets GITHUB_TOKEN env for adaptive relevance case.
tests/cases/renovate-deps/fast-only-irrelevant/test.toml Sets GITHUB_TOKEN env for adaptive relevance case.
tests/cases/renovate-deps/ci-missing-token/test.toml Adds CI case asserting renovate-deps fails without required token(s).
tests/cases/renovate-deps/ci-missing-token/files/renovate.json5 Adds minimal renovate config fixture for CI-missing-token case.
tests/cases/renovate-deps/ci-missing-token/files/mise.toml Adds renovate tool fixture for CI-missing-token case.
tests/cases/lychee/local-missing-token-warns/test.toml Adds local case asserting lychee warns (not fails) when token missing.
tests/cases/lychee/local-missing-token-warns/files/mise.toml Adds lychee tool fixture for local-missing-token case.
tests/cases/lychee/local-missing-token-warns/files/lychee.toml Adds minimal lychee config fixture.
tests/cases/lychee/local-missing-token-warns/files/README.md Adds fixture content for lychee scan.
tests/cases/lychee/clean/test.toml Adds token + fake bin to make lychee case deterministic.
tests/cases/lychee/ci-pr-missing-remap-env/test.toml Adds CI PR case asserting missing PR remap metadata fails in diff mode.
tests/cases/lychee/ci-pr-missing-remap-env/files/mise.toml Adds lychee tool fixture for CI PR remap-metadata case.
tests/cases/lychee/ci-pr-missing-remap-env/files/lychee.toml Adds minimal lychee config fixture.
tests/cases/lychee/ci-pr-missing-remap-env/files/README.md Adds fixture content for lychee scan.
tests/cases/lychee/ci-pr-missing-remap-env/changes/README.md Adds changed-file fixture for diff-mode lychee scan.
tests/cases/lychee/ci-missing-token/test.toml Adds CI case asserting lychee fails without GITHUB_TOKEN.
tests/cases/lychee/ci-missing-token/files/mise.toml Adds lychee tool fixture for CI-missing-token case.
tests/cases/lychee/ci-missing-token/files/lychee.toml Adds minimal lychee config fixture.
tests/cases/lychee/ci-missing-token/files/README.md Adds fixture content for lychee scan.
tests/cases/lychee/ci-full-missing-remap-env/test.toml Adds CI full-mode case asserting PR remap metadata is not required under --full.
tests/cases/lychee/ci-full-missing-remap-env/files/mise.toml Adds lychee tool fixture for CI full-mode case.
tests/cases/lychee/ci-full-missing-remap-env/files/lychee.toml Adds minimal lychee config fixture.
tests/cases/lychee/ci-full-missing-remap-env/files/README.md Adds fixture content for lychee scan.
tests/cases/lychee/broken-link/test.toml Adds token + fake bin output for deterministic broken-link behavior.
tests/cases/general/init-rust/test.toml Updates expected init output ordering due to refactored init hook generation.
tests/cases/general/fast-only-explicit-override/test.toml Updates expected stderr to include renovate-deps missing-token warning.
src/runner.rs Replaces special-check handling with a unified native-check preparation/run path; adds hooks for output normalization and missing-component hints.
src/registry/types.rs Introduces CheckType, NativeCheck, PreparedNativeCheck, InitHookContext, and additional registry metadata fields (relevance hooks, status hooks, baseline triggers, etc.).
src/registry/tests.rs Updates registry tests for native checks and adds assertion that adaptive checks declare relevance hooks.
src/registry/mod.rs Re-exports new registry types/hooks used across the codebase.
src/registry/checks.rs Migrates former special checks to native checks; wires check-type metadata, adaptive relevance, baseline triggers, and workflow setup flags.
src/main.rs Switches adaptive fast-only gating to registry-owned relevance hooks and replaces special-kind checks with native/setup checks.
src/linters/yamllint.rs Adds check-type init hook to generate .yamllint.yml.
src/linters/taplo.rs Adds check-type init hook to generate .taplo.toml and provides non-verbose failure output normalization.
src/linters/rustfmt.rs Adds check-type init hook to generate rustfmt.toml.
src/linters/rumdl.rs Moves rumdl config generation and legacy conversion logic into linter module as an init hook.
src/linters/renovate_deps.rs Adds native check implementation, CI/local env validation, adaptive relevance hook, flint.toml/preset patching init behavior, and token forwarding behavior.
src/linters/mod.rs Adds new linter modules and re-exports LinterOutput from registry.
src/linters/lychee.rs Adds native check implementation plus CI/local env validation for tokens and PR remap metadata.
src/linters/license_header.rs Converts license-header into a native check with status override and uses shared file matching helper.
src/linters/flint_setup.rs Converts flint-setup into a native setup check with tracked-files support.
src/linters/env.rs Adds shared CI/token env helpers (truthy CI detection, token checks, warning messages).
src/linters/biome.rs Adds check-type init hook to generate/migrate biome.jsonc.
src/init/tests.rs Adds coverage ensuring shared check-type init hooks run once per type; updates tests to use new generator locations.
src/init/scaffold.rs Renames workflow generation flag to reflect rust-components setup needs.
src/init/renovate.rs Removes old renovate patching module (logic moved into renovate-deps linter/init hook).
src/init/mod.rs Refactors init to run check-type init hooks and determine workflow setup needs from registry metadata.
src/init/generation.rs Removes re-exports tied to deleted renovate module.
src/init/config_files.rs Simplifies flint.toml generation (renovate-deps section now handled by check-type init hook).
src/files.rs Makes FileList Debug and moves match_files/glob helper into shared module for reuse by native checks.
src/config.rs Updates comments to refer to native checks instead of special checks.
docs/linters.md Updates terminology and docs for native checks; documents new CI token/PR-metadata requirements for lychee and renovate-deps.
docs/cli.md Updates baseline behavior docs to refer to native checks.
README.md Updates GitHub env documentation to reflect per-check requirements and points to lychee docs for PR remap env.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/linters/renovate_deps.rs Outdated
@zeitlinger
fix: avoid invalid renovate extends patch for empty config
72aad8b 
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
This was referenced Apr 29, 2026
Merged
Closed
Merged
Merged
Closed
Merged
@zeitlinger zeitlinger merged commit 5f57fb8 into main Apr 30, 2026
13 checks passed
@zeitlinger zeitlinger deleted the codex/ci-env-required branch April 30, 2026 12:59
zeitlinger added a commit that referenced this pull request Apr 30, 2026
@zeitlinger
fix: run renovate-deps for deleted tracked files (#247)
9ac7302 
Blocked by #253


## Summary

Fixes adaptive `renovate-deps` relevance when the only relevant change
is deleting a file that is still tracked in
`.github/renovate-tracked-deps.json`.

The changed-path collection now preserves deleted paths, and
`renovate-deps` uses the raw changed path list for relevance instead of
only existing files.

## Validation

- `cargo fmt -- --check`
- `git diff --check`
- `cargo test renovate_deps`
- `FLINT_CASES=renovate-deps/fast-only cargo test cases`

Note: `FLINT_CASES=renovate-deps cargo test cases` was attempted, but
non-fake renovate fixtures need `renovate` installed on `PATH`.
zeitlinger added a commit that referenced this pull request Apr 30, 2026
@zeitlinger
chore(renovate): simplify quickstart and batch weekly linter updates (#…
e4dd59e 
…257)

Blocked by #253

## Summary

- shrink the README quickstart `[tools]` example to three representative
linters
- manage those three README example versions with repo-local Renovate
regex managers
- keep the shared `default.json` free of repo-specific README rules
- update the Renovate drift test so only shared managers are compared
against `default.json`
- keep flint-managed linter updates in a single weekly Monday Renovate
PR with a clearer generated title

## Why

The previous approach made README quickstart version drift break
Renovate PRs, even though the README example is repository-specific.
This change keeps the example small, keeps its version bumps automatic,
and avoids pushing repo-local behavior into the shared preset.

The follow-up Renovate change also keeps linter bumps batched into one
Monday PR, including majors, so the repo gets one coordinated
linter-update PR instead of split grouped PRs with generic titles.

## Validation

- `renovate-config-validator --no-global .github/renovate.json5`
- `cargo test
repo_renovate_config_stays_aligned_with_shared_preset_contract --
--nocapture`
- `cargo test readme_quickstart_tools_snippets_stay_current --
--nocapture`
- `cargo test default_renovate_preset_covers_all_linter_tools_weekly`
- `cargo test
repo_renovate_config_stays_aligned_with_shared_preset_contract`
- `mise run lint:fix`
zeitlinger added a commit that referenced this pull request May 1, 2026
@zeitlinger
fix(init): let rustfmt own Rust line length (#250)
8acb866 
Blocked by #253


## Summary

When Rust formatting is enabled, `flint init` now rewrites existing
matching `.editorconfig` sections so Rust line length clearly defers to
`rustfmt`.

Previously an existing `[*.rs]` section with `max_line_length = 100`
survived unchanged and could conflict with generated `rustfmt.toml`
policy.

## Validation

- `cargo fmt --check`
- `cargo test init::tests::disable_editorconfig_line_length_for_patterns
-- --nocapture`
- `cargo test init::tests:: -- --nocapture`
- `FLINT_CASES=general/init-rust cargo test cases -- --nocapture`
- `git diff --check`
This was referenced May 8, 2026
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@martincostello martincostello martincostello approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zeitlinger @martincostello